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i— —I- Abstract 

H; 

, Hanson's market scoring rules allow us to design a prediction market that still gives useful informa- 

\ tion even if we have an illiquid market with a limited number of budget-constrained agents. Each agent 

can "move" the current price of a market towards their prediction. 

While this movement still occurs in multi-outcome or multidimensional markets we show that no 
market-scoring rule, under reasonable conditions, always moves the price directly towards beliefs of the 
(N. agents. We present a modified version of a market scoring rule for budget-limited traders, and show that 

£ — . it does have the property that, from any starting position, optimal trade by a budget-limited trader will 

result in the market being moved towards the trader's true belief. This mechanism also retains several 
\ attractive strategic properties of the market scoring rule. 

1 Introduction 

Prediction markets are markets set up for the primary purpose of aggregating information to forecast future 
events. For a heavily traded prediction market, such as whether Obama will be reelected in 2012, we can 
get a very precise prediction from the market. As of January 30, 2012, the Intrade bid-ask spread gives a 
prediction between 53.3% and 53.9% of reelection. For other less liquid markets, the bid-ask spread gives a 
less informative prediction. The Intrade bid-ask spread for whether Sarah Palin will endorse Mitt Romney 
is sitting between 15.1% and 99.0%. 

To recover information in thin, illiquid markets, Robin Hanson developed the market scoring rule [Han03 ]. 
With this form prediction market, a market maker always quotes a single current price for each future events. 
If the agents have a belief different than the current price, a trade will give them an a priori expected positive 
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profit. The market maker adjusts the price as trades are made. If traders are risk-neutral and do not face 
budget constraints, Hanson showed that it is optimal for a trader to trade until the price matches her beliefs; 
the final market prices can thus be interpreted as updated probabilities for the forecast events. The market 
maker loses money in expectation, but the total loss of the market maker can be bounded. 

The further the change in price, the higher the up-front cost, and worst-case loss, that the trader has 
to bear. In real-world prediction markets, traders often have budget constraints that limit how much loss 
they can bear. In fact, many real money prediction market platforms impose limits on how much money 
an agent can invest over the entire platform, and most play-money markets limit the ability of traders to 
convert external money to virtual points that can be invested. Thus, limited budgets are an inherent feature 
of the context in which most prediction markets are run. If an agent is budget-limited she can still trade, but 
may not be able to move the prices to match her beliefs. For two-outcome markets, like the ones described 
above, the market scoring rule induces straightforward behavior in budget-limited agents: An agent will 
move the prices of the securities directly towards her belief, and the final probability forecast will therefore 
be a mixture of the original market probabilities and the agent's true belief. In this paper, we explore the 
behavior of budget-limited traders in markets with multiple outcomes or multi-dimensional outcomes. 

Many natural events for which we seek forecasts have multiple outcomes. Consider the following sce- 
narios: 

• A market over Republican candidates for the U.S. Presidential nomination: For this market, a forecast 
can be represented by a four-tuple probability vector (pr,Pg,Pp,Ps) f° r the probabilities that Romney, 
Gingrich, Paul and Santorum will be nominated. 

• A market predicting the latitude and longitude of the next tornado strike in Kansas. Here, the forecasts 
consist of probability distributions over a two-dimensional grid. 

• A market predicting the likelihood of hurricane strikes for each year over the next three years. Here, 
a forecast consists of a probability distribution over {0, l} 3 ; however, it may be natural to assume 
independence between different years. 

For each of these markets, we can set up prediction markets based on proper scoring rules. One way to 
do so is to break the markets into their components, a separate market for each candidate, for latitude and 
longitude and for each year. More generally, however, we may have a complex combinatorial market whose 
payoffs are dependent on the combination of events that occur [Han03 ]. In either case, a risk-neutral trader 
without budget constraints would optimize her payoffs by moving the market forecast to match her true 
belief. When traders are budget-limited, the optimal behavior may depend on the particular proper scoring 
rule used. One natural goal is to ask: Can we find a proper scoring rule such that, for any positive budget an 
agent has, her optimal trade is always to move directly towards her true belief, so that the updated forecast 
is a mixture of the original forecast distribution and her true belief? 
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Our main results show that this is impossible. In section[3]we show that, under mild smoothness assump- 
tions, no scoring rule over three or more outcomes can have this property. For example, in the Republican 
candidate setting, we cannot find any market scoring rule that will induce a budget-constrained trader to 
move the market forecast in a straight line towards her true belief. Further, we show that there are neigh- 
borhoods of possible beliefs that all result in exactly the same report; this shows that sequential aggregation 
in the MSR with budget constraints can be imperfect, because even if all budgets are public information, 
a trader's belief cannot be inferred from her report. The results in section [3] relies on the space of beliefs 
being unrestricted, and thus seems to leave open the possibility of a positive result when there are natural 
restrictions, such as independence of hurricane probabilities in different years. However, we show that this 
is not the case in section |4j Even when the outcome space consists of two binary events that are assumed 
to be independent, no smooth market scoring rule can induce a budget-limited agent to move the market 
distribution directly towards her true belief. Importantly, this latter result holds without assuming that the 
scoring rule is itself simple or separable, i.e., it may be a combinatorial scoring rule. 

In section [5J we present a modified market scoring rule, the Scaled Scoring Mechanism, that has this 
property of inducing budget-limited agents to move the forecast directly towards their true belief. This 
mechanism requires a trader to report her budget as well as her belief; we assume that the budget must be 
deposited up-front, and hence cannot be exaggerated (although it may be under-reported). We prove that a 
trader's optimal trade is to report her true budget and belief, and hence, the updated market forecast is always 
a mixture of her true belief and the original forecast. Further, we show that this mechanism retains many 
of the desirable properties of the market scoring rule: The market-maker's loss is bounded, and the traders' 
profit along a sequence of updates is sub-additive, so that a trader cannot gain a profit from splitting up her 
trade into two or more successive trades by different identities. As this mechanism induces full information 
revelation from budget-limited traders, it also enables effective sequential aggregation of information: A 
trader who has seen previous traders' trades can infer their beliefs; hence, in a Bayesian world, a trader can 
generically condition her beliefs on all previous traders' private information. 

The rest of this paper is structured as follows: We discuss related work in section [LT1 We introduce 
our formal model in section [2] We prove the impossibility results in section [3] and [4] and introduce the 
new mechanism in section [5] Finally, in section [6j we discuss extensions and limitations of our results, and 
directions for future work. 

1.1 Related Work 

There is a long literature on the use of proper scoring rules to incentivize accurate probabilistic forecasts 
of events [Goo52] [WM68]. Proper scoring rules are characterized by the property that, given any belief 
p, the report that globally maximizes the forecaster's expected (under belief p) reward is the honest report 
q = p. Subsequent research on effective scoring rules studied the design of scoring rules with a stronger 
monotonicity property. A scoring rule is said to be effective with respect to a given metric L over the space 
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of probability distributions if, given any two possible reports q,q' such that q is closer (under metric L) 
to the true belief p' than q', the expected score of reporting q must be greater than the expected score of 
reporting q' [Fri83]. Friedman [Fri83] has shown that there are scoring rules that are effective with respect 
to some metrics, including the L2 metric. Our negative results present a striking contrast: We show that no 
scoring rule can induce a budget-limited agent to move only along the direction of steepest decrease in L2 
distance to the true belief. 

Hanson's market scoring rules [Han03 ] transform scoring rules into a mechanism for obtaining sequen- 
tial forecasts from multiple forecasters. Hanson showed that this sequential scoring form can serve as an 
automated market maker for prediction markets. 

Budget limits can be viewed as a special form of risk aversion. Allen [A1187] described a lottery 
technique to extend scoring rules to forecasters with arbitrary and unknown risk aversion. Dimitrov et 
al. HDSE091 show that this technique can be extended to a sequential forecasting mechanism. However, the 
mechanism of Dimitrov et al. has the undesirable property that expected profits shrink exponentially as the 
number of traders grows, and moreover, they prove that this is unavoidable in the context of mechanisms for 
arbitrarily risk averse agents. The mechanism we present here does not have this undesirable property. 

Manski [Man04] and Wolfers and Zitzewitz [ WZ06I have also studied the effect of budget limits on the 
aggregative properties of prediction markets. However, our model differs from their models in significant 
ways: They study a stylized unsubsidized market model in which clearing prices are determined in a one- 
shot game, whereas we model a market scoring rule with commonOknowledge Bayesians who update beliefs 
after each trade. 

2 Model 



Outcomes and Distributions: Consider an event with more than two possible outcomes. In order to 
simplify the notation, we begin with a three-outcome event, and let the outcomes be denoted X,F,Z. An 
agent has a belief p = (px,py,Pz) on the probabilities of the various outcomes. When asked for a forecast, 
the agent may report a forecast q = (qx,lY,Qz)- In general, for unrestricted beliefs over a ^-outcome event, 
p and q can be represented as ^-dimensional vectors, with the implicit constraints that the elements are 
non-negative and sum to 1 . 

We also consider a special, natural, class of restricted distributions, which arise when the event being 
forecast itself has a dimensional structure. For clarity, we defer the description of this restricted class to 
section |4] 

Scoring Rules The agent is rewarded based on a scoring rule S. We model S as a mapping from the space 
of forecasts q to the space of payments S(q) = (5x(q),5y (q),5z(q)) that will be made for each outcome. 
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For unrestricted beliefs over three-outcome events, the space of all probability distributions A = A({X,Y,Z}) 
is a two-dimensional space. The scoring rule S can be viewed as a two-dimensional surface in 9^ 3 ; each point 
S(q) £ 9t 3 corresponds to a particular value of q. 

Assumptions: We assume that the scoring rule S(q) is continuous and differentiable over the interior of 
the space of possible reports q. (Actually, all we really need is that it is continuous and differentiable 
over some open set of reports.) We also assume that each component of the scoring rule is quasiconcave: 
Vp,q 5x(0.5p + 0.5q) > min(5x(p),5'x(q), with equality only if Sx(p) = Sjf(q); an ^ likewise for Sy,Sz- 
This is a mild condition (weaker than monotonicity of the score functions), that is true for all commonly 
used scoring rules. It is used to argue that, when moving from q to p, the budget increases monotonically 
along the path, and so any point on the path is optimal for some budget. We assume that agents are risk 
neutral (up to their hard budget constraints, described below); thus, an agent's goal is to maximize her 
expected score p • S(q). 

Market Scoring Rules and Sequential Updates Hanson's market scoring rules (MSR) HHanQ31 are a 
form of prediction market that extends scoring rules to a sequential information aggregation mechanism. 
Consider a set of agents {1,2, ... ,m} making sequential forecasts qi,q2, ■ • ■ ,q m - With the market scoring 
rule based on scoring rule S, each agent i receives a net payoff equal to the difference between the score of 
q, and q,_i. Thus, the net expected payoff of an agent i who believes a distribution p is: 

p.[S(q I -)-S(q ! -_ 1 )] 

Here, qo is an initial default distribution specified by the mechanism. 

Hanson observed that, with the MSR mechanism, if each agent trades just once and there are no budget 
constraints, the optimal action for each trader is to report q, that matches her true belief. Consequently, 
future traders who can observe the past reports (prices) in the market can condition on all previous agents' 
beliefs about the event. Generically, this leads to perfect aggregation of the m agents' combined information. 

In our analysis of scoring with budget constraints, each agent's report q is scored with respect to a 
reference point q, and thus her score is given by the difference S(q) — S(q). The notion of a reference point 
or default q is an essential ingredient of a model with budget constraints, because it describes what the 
forecast will be when an agent has budget. The reference point may be the previous traders' report q,_,, as 
in the market scoring rule, but it may also be determined differently; in section [5] we present a mechanism 
with a different way of determining a reference point for each trader. 

Budgets and Constrained Effectiveness We are interested in scoring rules that are strictly proper: The 
unique optimal q, given belief p, is q = p. However, we are further interested in a budget-limited version of 
this property. Informally, we want the scoring rule to have the property that, given any initial value q, and 
any specified budget &,fhe optimal report within the budget feasible set is a report q along the path from q 
to p. 
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In order to make this more rigorous, we need to define the notion of a budget more formally. One natural 
definition of the budget required to move from q to q is in terms of the maximum loss that the agent can 
incur from such a move; this is the amount of money that an agent would require to have to avoid defaulting 
on the mechanism. 

Definition 1 The natural budget nb(q, q) required to move from q to q, given a scoring rule S, is defined 
by 

nb(q,q) "= max [p • (S(q) - S(q))] = max{S x (q) - S x (q),S Y (q) - S Y (q),S z (q) - S z (q)} 
p 

Together, the reference point q and an initial budget holding b determine the range of possible reports that 
an agent can make while avoiding default under any outcome; the natural budget constraint determines the 
set of feasible reports. Thus, the natural budget constraint determines a budget-constrained agents' choice 
set under the market scoring rule as typically implemented. However, it may be possible for an alternative 
mechanism to further restrict the set of allowable reports by an agent; in section [5j we will exploit this to 
obtain a mechanism with better aggregative properties. 

Now, the natural budget-constrained optimal report q*, with budget b, is the choice among all q such 
that nb(q,q) < b, that maximizes p • S(q). Let q*(p,q,b) denote the optimal report with budget b. 

We can now state our desired property: 

Definition 2 A scoring rule S satisfies the budget-constrained truthfulness property if the budget-constrained 
optimal choice is always a mixture of the initial distribution q and the beliefs, and as close to p as possible: 

Vp,q,6 > q*(p,q,fc) = ap + (1 - a)q 

where 

a G [0, 1] = max{oc|6(q, ocp + (1 - oc)q) < b} 

This definition inherently implies that S is a proper scoring rule (by taking a sufficiently large b); how- 
ever, it is significantly stronger than the properness condition. 

3 An impossibility result for unrestricted multi-outcome distributions 

For two-outcome events, the budget-constrained truthfulness property holds trivially for all common scoring 
rules, as the natural budget and expected score both increase monotonically as q moves from q towards the 
agent's true belief p. However, well-known scoring rules do not satisfy this for general multiple-outcome 
events. Here, we show that this is not accidental: We will prove that no scoring rule (satisfying our technical 
assumptions) over three or more outcomes can satisfy the budget-constrained truthfulness property. We will 
assume without loss of generality that there are exactly three outcomes X,Y,Z; the result holds a fortiori for 
k>3. 
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We will prove the impossibility result by contradiction. Suppose that we had a scoring rule S such that 
S satisfied the budget-constrained truthfulness property. 

We first make a simple observation about the local structure of the scoring surface: 

Lemma 1 Consider a point p in the interior of A, and consider a neighboring point of the form p + 8e, 
where £ is a nonzero vector such that £^ + £y + £z = 0, and 8 is an infinitesimally small quantity. Then, 
there exists a direction E such that ^(j*" 1 " 86 ) = 0. Further, for this £, ^(p+fe) an j rfs z (p+fe) mmt ^ e nonzero 
and have opposite signs. 

Proof: Expanding Sx around p in terms of its partial derivatives, we have: 

dS x {p + Se) _ dS x dS x dS x 
do dp x dp Y dp z 

Setting the LHS to 0, we get one linear constraint on Ex and £y ; any solution to this will meet our purposes. 

For the second part of the statement, consider the point q = p + ed8. We have Sx (q) = S x (p). However, 
as this is a proper scoring rule, we must have p • [5(p) — 5(q)] > and q • [S(p) — 5(q)] < 0. As p and q 
have non-negative entries, the only way in which this can happen is if 5(p) — 5(q) has one positive and one 
negative entry. ■ 

We next establish an intuitive result: given two points p, q, there is a budget b such that the constrained- 
optimal report is the midpoint between p and q. 

Lemma 2 Consider a scoring rule S that satisfies the budget-constrained truthful property. For any pair 
p,q € A, there is a b such that q*(p,q,fc) = 0.5q + 0.5p. 

Proof: Let b = nb(q,0.5q + 0.5p). We must have b > 0, because if b < 0, then an agent who believed q 
would make at least as much expected profit by reporting 0.5q + 0.5p instead of being truthful; this would 
violate the properness of the scoring rule. 

Given the definition of the budget-constrained truthful property, we only need to show that, for any a > 
0.5, nb(q,ap + (1 -a)q) > b. Without loss of generality, let us assume that S x (q) -5x(0.5p + 0.5q) = b > 
0. Given a a > 0.5, we note that (0.5p + 0.5q) is a mixture of q and (ocp + (1 - oc)q. By our quasiconcavity 
assumption, it must be true that 5x(ap + (1 — oc)q < Sx(0.5p + 0.5q). But then, by the definition of the 
budget, we must have nb(q, ocp + (1 — cc)q) > b. ■ 

The following lemma will allow us to construct a contradiction: 

Lemma 3 Suppose that S is continuous, differentiable, and satisfies the budget-constrained optimality prop- 
erty. Then, there are probability distributions p,q,q,p',q' and budgets b,b' such that the following proper- 
ties are satisfied: 

1. q = 0.5p + 0.5q and q = q*(p,q,b). More specifically, 5x(q) — 5x(q) = b and 5y(q) — 5y(q) < b, 
S z (q)-S z {q)<b. 
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2. q = 0.5p' + 0.5q'a«<f q = q*(p',q',Z/). More specifically, S x (q') - S x (q) = b' and Sy(q') - S Y (q) < 
b',S z (q')-S z (q)<b>. 

? Ex. -A Ejl 
J - Pz ^ 7z 

Proof: We pick arbitrary p and q in the interior of the space of possible distributions, and let q = 0.5q + 0.5p 
be a mixture of these two distributions. Let b = nb(q,q). By lemma[2j q = q*(p,q,b). Now, the budget b 
is the worst-case loss of moving from q to q; without loss of generality, assume that this loss occurs with 
outcome X. Thus, we must have: 

S x (q)-S x (q)=b 

Suppose that we also have Sy (q) — Sy(q) = b. Then, it must be the case that Sz(q) < Sz(q)- By LemmaQ] 
we can perturb our choice of p slightly such that only S x (q) — S x (q) = b, while S Y (q) — Sy(q) < b and 
S z (q)-S z (q)<b. 

Now, consider q' = q — £ and p' = p' + £, where £ is a small perturbation that is constrained to ensure 
that q' and p' are still probability distributions. Note that q = 0.5q' + 0.5p'. By lemma [2j there is some 
budget b' such that q = q*(p / ,q',Z7 / ). By the continuity of S, for all £ within a sufficiently small ball, it must 
still be true that S x {q) - S x (q) > Sy(q) - 5 y (q),5 z (q) - 5 z (q). 

To get the last part of the construction, we pick £ such that p' Y > py,p' z < pz- ■ 

Finally, we can use the constructed structure of Lemma[3]to prove a contradiction. 

Theorem 4 There is no scoring function S that satisfies the technical conditions (continuity, differentiability, 
and quasiconcavity) and also satisfies the budget-constrained truthfulness property. 

Proof: Suppose that there is such a scoring function. Consider p,q,q,p',q / and budgets b,b' as constructed 
in Lemma [3] 

By assumption, if the initial position is q, and an agent has belief p and budget b, the optimal report is 
q. With this report, the agent's expected score is p • 5(q) — p • 5(q). 

Now, consider the neighborhood of the report q. By Lemma [T] there is a direction £ such that 5x(q + 
edb) = S x (q). By lemma[3l and the continuity of S, this means that nb(q, q +£<f 8) = nb(q, q) = b, as locally 
S x is the binding constraint on the budget. In other words, the report q + edb, as well as q — edb, are in the 
feasible set for an agent with budget b. As S x is constant along direction £, the factors affecting the agent's 
expected score are the change in Sy and Sz- The first-order conditions for optimality at q then give us: 

dSy (q + E§) <i5 z (q + £8) 
P¥ +PZ =° 

Rearranging, we must have: 

dS z (q+tS) 

d& = _PY_ 
rfSr(q+eS) pz 
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Note that the left hand side depends only on q. Therefore, if we repeat our analysis with p', q', and b' we 
would derive that the same LHS (with the same vector e) is equal to — Of-. By the last assertion in Lemma|3j 

Pz 

this is a contradiction. ■ 

Thus, for unrestricted distributions over 3 or more outcomes, it is impossible to find a smooth scoring 
rule that will always incentivize agents to move towards their true belief, given the natural budget constraint. 

In fact, we can prove an even stronger negative result about the aggregative properties of the market 
scoring rule in the presence of hard budget constraints: We can show that the optimal report of a budget- 
constrained agent can be locally insensitive to changes in her belief p. This implies that future traders may 
not be able to infer this trader's belief from her trade, thus resulting in imperfect aggregation of information 
even if all budgets are known. 

In order to prove this result, we first prove two technical lemmas. The first result just shows that there is 
a reports r such that two budget constraints are tight. 

Lemma 5 Given a strictly proper scoring rule S that satisfies the technical assumptions ( differentiability 
and quasi-concavity), and given any starting probability distribution q in the interior of the space of distri- 
butions over {X,Y,Z}, there is a distribution r and constants a,b > such that two of the budget constraints 
with budget b are tight: 

Sx(r)-S x (q) = -b;S Y (r) - S ¥ (q) = -b;S z (r) -5 z (q) = a 

Proof: Consider the set of all distributions as two-dimensional space. As q is an interior distribution, there 
is a circle Q = {q} of distributions such that |q — q| = c. Consider the function nb(q, q) on Q. As Q is a 
compact space, and nb is continuous, this function must have a minimum value, call it 2b, that is achieved 
within Q. As this is a proper scoring rule, we must have 2b > 0. 

Now, consider some distribution r' such that nb(q,r') = b. Thus, r' must lie within the circle Q. Without 
loss of generality, let us suppose that Sx(r') — &(q) = —b. Then, by lemma [T] there is a direction of 
movement around r such that Sx(-) is constant, while Sy(.) decreases. Moving along this direction, we must 
reach a point r at which both Sx(r) — Sx(q) = —b and Sy(r) — Sy(q) = —b. At this point r, we must have 

def _ _ 

a = 5z(r) — 5z(q) > 0, because otherwise q would earn a higher score than r for all three outcomes, which 
is impossible in a proper scoring rule. ■ Next, we show that for beliefs in the neighborhood of such a point 

r, we can find a 2-dimensional open set with certain properties: 

Lemma 6 Given the construction ofLemma\5\ consider perturbations to the distribution r parameterized 
by a pair (Ejr,Ey). Define £z = + £y) ond consider distributions p = r + £. Then, there is an open 
neighborhood N of pairs (&x , By ) (or equivalently, a neighborhood of distributions p ) such that the following 
conditions hold: 

1. Zx < 0,£y < 0,£ Z > 
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2. 5 z (p) >S z (r)-a 

3. Either Sx(p) < Sx(r) or Sy(p) < Sy(r), or both. 

Proof: Consider the set N\ of all e such that condition (1) is satisfied, and £ is within a small enough 
ball that p is a valid distribution; this set is an open set. Now, consider all £ within a small ball such that 
Sz(p) > 5z(r) — a - Because the scoring rule is continuous, this set must include an open set N2 around 
£ = (0,0,0). Now, Ni and N2 have a non-empty intersection N = N\ PiNzl N is also an open set. 

Thus, it remains to show that, for all £ in N, condition (3) is satisfied. We prove this by contradiction. 
Suppose that there was sl£ £ N such that condition (3) was not satisfied, i.e., Sx(p) > Sx{r) and 5y(p) > 
5y(r). We must have Sz(p) < Sz(r). Then, we would have: 

(p-r).[S(p)-S(r)] = £.[S(p)-S(r)]<0 
=>p.[S(p)-S(r)] < r.[S(p)-S(r)] 

The first inequality follows from the constructed signs of £z,£y, and Zz- However, for a proper scoring 
rule, the left hand side of the last inequality is positive, while the right hand side is negative, leading to a 
contradiction. ■ 

Now, we can show that, for all beliefs p in this 2-dimensional open set N, the optimal report would be 
the same, r. 

Theorem 7 Given any proper scoring rule satisfying the technical assumptions, and any interior starting 
position q, there is a 2-dimensional open set N of beliefs p, a budget b, and a feasible report r such that, for 
all beliefs in N, the optimal budget-constrained report is r. 

Proof: Construct N,b, and r according to Lemmas [5] and Lemma [6] Consider p in N. We can see, by the 
third condition in Lemma [6l that p is not a feasible report given budget b. 

Now, consider any feasible report q/r. As q is feasible, we must have 5x(q) > Sx{r) and 5y(q) > 
5y(r). As S is a proper scoring rule, it follows that we must have 5z(q) < 5z(r). Now, for p and £ = p — r, 
we have: 

(p-r).[S(q)-S(r)] = e.[S(q) - S(r)] < 
=»p.[S(q)-S(r)] < r.[S(q)-S(r)] 

As S is proper, the RHS of the last inequality is negative. Thus, the LHS must also be negative, implying 
that, under belief p, r gives a higher expected score than q. As this is true for all feasible q, r must be the 
optimal report. ■ 

This result shows that failure of information aggregation cannot be avoided with the market scoring rule, 
even if all budgets are common knowledge: For an open set of beliefs, the corresponding report is the same 
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r, and hence future traders cannot infer or condition on the precise belief held by a trader. For forecasting 
problems with k > 3 outcomes, we believe that it should be possible to prove an extension to theorem [7] that 
shows that the report is insensitive to beliefs in a (k — 1) -dimensional neighborhood; this is a direction for 
future work. 

4 Impossibility results for multi-dimensional events 

For some multi-outcome events, we may have reasonable constraints on the family of distributions that 
agents may believe and report. Of course, there are innumerable different ways of defining constrained 
distributions over the outcomes. In this section, we consider one very natural form of constraint that arises 
when the outcome itself can be decomposed into independent dimensions. For example, when eliciting 
beliefs about the likelihood of a hurricane striking a city this year and next year, it may be natural to assume 
that, although there are four possible outcomes overall, the likelihood of a strike is independent in the two 
years, and as such, we can elicit independent probabilities for each year. In this section, however, we show 
that Theorem |4] extends to this class of restricted preferences as well. 

Without loss of generality, we focus on a model with two dimensions of outcome and two possibilities in 
each dimension. (The impossibility result directly follows for richer outcome and report spaces.) Suppose 
that the outcome consists of a pair, with the first component either Top or Bottom, and the second component 
either Left or Right. Each forecast q is also a pair (#r,?z.)- m trns model, we assume that the events Top 
and Left are independent, so that these two parameters suffice to determine the outcome. We will use the 
notation q to denote the vector (<7r<?Lj<7r(l — (1 — <?r)^L ; (l — <7r)(l — qi)), i-e., the probabilities of 
(TL, TR,BL,BR) implied by a report vector q = (qr,lL)- 

Given the set of pairwise outcomes {TL,TR,BL,BR}, we can represent a scoring rule as a set of four 
functions: 

S(q) = [S T L(q),STR(q),SBL(q),S B R(q)] 

Note that we do not assume that the scoring rule is separable into separate scoring rules for each dimension. 
An agents' score may depend arbitrarily on the pair of outcomes. 

Theorem |4] does not immediately imply an impossibility in this domain, because not all distributions 
over {TL, TR,BL,BR} are expressible in this model. The set of reports is two-dimensional, A = A({T,B}) x 
A({L,R}). (If we had not assumed independence of Top and Left events, but instead collected a separate 
probability for each of the four outcomes, then Theorem 0] would immediately apply.) However, we will 
show below that a very similar construction works in this case as well. 

We first prove an analogue of Lemma \T\ 

Lemma 8 Consider a point p in the interior of A, and consider a neighboring point of the form p + 86, 
where £ is a nonzero vector such that (Zj , £/,), and 8 is an infinite simally small quantity. Then, there exists 
a direction £ such that ^ rL ^P +Se ) = 0. Further, for this £, out of the three other directional derivatives 
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dSrR ^^ t ds Bt{p+&) ^ anc i </Sbr (,p+8e) ^ af j east one muS f b e strictly positive, and at least one must be strictly 
negative. 

Proof: Expanding Sx around p in terms of its partial derivatives, we have: 

dSx(p + te) dftx | dS TL 
dS dpr dpL 

Setting the LHS to 0, we get one linear constraint on 87- and 8^; any solution to this will meet our purposes. 

For the second part of the statement, consider the point q = p + £<f8. We have SjxCq) = Stl(p)- How- 
ever, as this is a proper scoring rule, we must have p • [S(p) — S(q)] > and q • [S(p) — S(q)] < 0. As p and 
q have non-negative entries, the only way in which this can happen is if S(p) — S(q) has at least one strictly 
positive and at least one strictly negative entry. ■ 

Lemma [2] does apply in this setting. Therefore, we move on to prove an analogue of Lemma |3j 

Lemma 9 Suppose that S is continuous, differentiable, and satisfies the budget-constrained optimality prop- 
erty. Then, there are probability distributions p,q, q = 0.5p + 0.5q, a ball radius r > and a budget function 
b(t) defined for all t : |t| < r such that the following property is satisfied. Let q(t) = q — t and p(t) = p + 1. 
Then, Vt s.t. |t| < r, 

q = q*(p(t),q(t),fe(t)) 

and 

5 ri (q(t)) -S TL (q) = b(t) > max{ 5 rJt (q(t)) -S TR (q), 

S BL (q(t))-S Bi (q),S fiR (q(t))-S B *(q)} 

Proof: We pick arbitrary q in the interior of the space of possible distributions, and pick a q sufficiently 
close to q such that q + (q — q) and q + (q — q) are both valid probability distributions. Let p = q + (q — q) , 
so that q = 0.5q + 0.5p. Let b = b(q,q). By lemma[2l q = q*(p,q,b). Now, the budget b is the worst-case 
loss of moving from q to q; without loss of generality, assume that this loss occurs with outcome TL. Thus, 
we must have: 

S T L(q)-S T L(q)=b 

Now, consider the other components of the score difference: (Sr#(q) — STR(q),SBL(q) — 5BL(q),^B«(q) ~ 
SsR{q))- We first show that we can perturb the choice of q (and perhaps permute the outcome names) such 
that all of these components are strictly less than b. 

Note that at least one of these components must be negative as the scoring rule is proper when the 
true distribution is q. If exactly one of these components is negative, we can swap q and q (changing p 
accordingly), such that there is a unique positive component among all four score differences, which would 
satisfy the requirement. 
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Thus, we can restrict our attention to the case in which one component - say Str (q) — Str (q) is positive, 
while the other two components are negative. In this case, there is a problem only if Sy#(q) — Str(<i) = b. 
Now, let us apply Lemma [8] This guarantees a direction of change such that Stl is unchanged. If Str 
changes along this direction, then we can slightly perturb q to get this property. If Str is unchanged, then, 
by lemma [8j both Sbl and Sbr must change, in opposite directions. In the latter case, we can again swap 
q and q; this would guarantee that exactly two components are positive, and a small perturbation would 
guarantee that one of them is uniquely the maximum. 

Now, we can safely assume that we have picked q and q such that the only tight budget constraint is that 
Stl (q) — Srz,(q) = b > 0. Consider q(t) = q — t and p(t) = p' + 1, where t is a small perturbation. By the 
continuity of S, for all t within a ball of sufficiently small radius r, it must still be true that S7x(q) — SrzXq) > 
5™(q) -5 rR (q),5 BL (q) -5 BL (q),5 BR (q) -5 BR (q). Note that q = 0.5q(t) +0.5p(t). By lemmaH there is 
some budget b(t) such that q = q*(p(t),q(t),Z?(t)). ■ 

Finally, we can use the constructed structure of Lemma [9] to prove a contradiction. This proof is analo- 
gous to Theorem @] 

Theorem 10 For the two-dimensional outcome setting, there is no scoring function S that satisfies the tech- 
nical conditions (continuity, differentiability, and quasiconcavity) and also satisfies the budget-constrained 
truthfulness property. 

Proof: Suppose that there is such a scoring function. Consider p(t),q,q(t),£(t) as constructed in Lemma|9] 
Fix an arbitrary t within the ball, and let us use the shorthand p,q,& for p(t),q(t),ft(t) respectively. By 
assumption, if the initial position is q, and an agent has belief p and budget b, the optimal report is q. With 
this report, the agent's expected score is p • 5(q) — p • 5(q). 

Now, consider the neighborhood of the report q. By Lemma[8j there is a direction £ such that SVzXq + 
edb) = Stl(<1)- By lemma|9j and the continuity of S, this means that b(q,q + Ed8) = b(q,q) = b, as locally 
Stl is the binding constraint on the budget. In other words, the report q + edb, as well as q — Ecf8, are in the 
feasible set for an agent with budget b. As Stl is constant along direction e, the factors affecting the agent's 
expected score are the change in Str, Sbl and Sbr- The first-order conditions for optimality at q then give 
us: 

„ _ / rfSr L (q+e8) dS T R(q+e$) dS BL (q+ES) dS BR (q+e§) \ = Q 
\ dd d5 dd dd J 

Note that q and £ are independent of our choice of t. Thus, for all t in an open neighborhood, we must 

have: 

p(t)-D = (1) 

where D is a vector with first component 0, and at least one positive and one negative component. In 
other words, all p(t) corresponding to an open set of reported probabilities p(t) must lie in a single linear 
subspace of 4-dimensional space. 
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We can now demonstrate a contradiction, by showing that, as the set of feasible p is curved, no linear 
subspace can hold an open neighborhood. Consider an arbitrary estimate po = (x,y) within this neighbor- 
hood, and three points in its neighborhood pi = (x + 8,y), P2 = (x,y + S), and P3 = (.x + 8,;y + 8), where 
8>0. 

Equation [TJ must then hold for ^0)Pi>p2,P3- Thus, we must get: 

(pi- Po )-D = 5(y,\-y,-y,-\+y)-D = 
(p 2 -p )-D = 8(x,-x,\-x,-\+x)-D = 

(p 3 -pi)-D = => 8(x + S,-jc-S,l-x-S,-l+;t + 8)-D = 

(subtract second eq from third) (1,— 1,-1,1) - D = 

(first eq. - by x fourth eq.) => (0, 1 , 0, - 1 ) • D = 

(second eq. - 8* x fourth eq.) =4> (0,0,1,— 1 ) • D = 



Collecting all the equalities, we have: 

(1,0,0,0)-D =0; (1,-1,-1,1)-D = 0; (0,1,0,-1) -D = 0; (0,0, 1,-1) -D = 

(The first equality is because, by construction, the first component of D is 0.) However, the only solution to 
this set of equations is D = (0,0,0,0). This contradicts the property, guaranteed by Lemma [8j that D has at 
least two nonzero entries. ■ 



5 A mechanism for budget-constrained elicitation 

In this section, we describe a mechanism, the Scaled Scoring Mechanism, for sequential information ag- 
gregation that modifies the market scoring rule to address the limitations under budget constraints. The 
mechanism and its characterization are fairly simple, but it serves to illustrate how designing for budget 
constraints can be useful. The sequential update and scoring rules we use were previously proposed for 
binary outcomes in the context of recommender systems by Resnick and Sami [RS07]; that paper did not 
address the properties for multi-outcome events or the possibility of mis-reporting budgets that we focus on 
here. 

We make one important assumption: Our mechanism will ask users to report their current budget b, but 
we assume that it is impossible for an agent to report a higher budget than her true budget b, although users 
may strategically under-report b. Such a mechanism could be sustained by requiring each agent to deposit 
her entire reported budget b up front, at the time of making her report. If the budget is truly a hard constraint, 
then an agent will not be able to deposit more than b, even if she is sure that she cannot lose all that she 
deposits. 
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Consider a sequence of agents 1,2, ... m interacting with a mechanism, in which the goal of the mecha- 
nism designer is to elicit an unrestricted distribution q over a given outcome space. Let S be a proper scoring 
rule that satisfies the following properties: Each component of S(q) is concave in q, and for all allowed q, q, 
nb(q, q) < B. In other words, the maximum budget required for any feasible move is bounded by B. For 
example, S may be the quadratic scoring rule. If the range of allowed distributions is restricted slightly so 
that all individual probabilities are bounded away from 0, we could even use the logarithmic scoring rule 
S(q) = (log q x , log q y , log q z ). 

The Scaled Scoring Mechanism operates as follows: 

1. At any time, the mechanism has a forecast q that also serves as a reference point. Initialize q = qo, 
an arbitrary initial default prediction. 

2. For i = 1,2, . . . ,m: 

(a) Ask agent i to report a budget (denoted by bfi and a forecast q ; . 

(b) Update 



3. When the outcome is revealed (say, to be X), score the agents as follows: each agent i gets a net payoff 



We make the following observations about the scaled scoring mechanism: Firstly, when reported budgets 
are unlimited {b\ > B), the updates and scoring are exactly the same as the market scoring rule. However, 
when budgets are limited, the score is a scaled-down value proportional to the market scoring rule score, 
and the reference point is moved only a fraction of the distance to the reported belief q,-. 

We can prove the following properties about the Scaled Scoring Mechanism: 

Theorem 11 The Scaled Scoring Mechanism satisfies the following properties: 

1. Myopic strategyproofness: Assuming that each agent i cannot report b\ higher than her true budget 
hi, and that each trader trades just once, it is optimal for each trader to report her true belief p* and 
her true budget 

2. No default: Under any outcome, each agent i's net payoff is at least —b\. 

3. Budget-constrained truthful: After i selects her optimal report, the forecast distribution q ; is a mixture 
of the earlier forecast distribution q ( _j and i's true belief p„ for any budget bj. 

4. Limited Loss: The worst-case loss of the market maker is no worse than the worst-case loss of a 
market scoring rule market for the same outcome space, with the same initial distribution q . 




equal to: 
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5. Myopic sybilproofness: It is never profitable for an agent to divide her budget among multiple identi- 
ties who make consecutive reports)^ 

Proof: ([I]): First, we observe that for any fixed reported budget b\, the optimal report q, is equal to agent 
fs belief p,: Her payoff is proportional to S(q ( ) — S(q,_i), and as S is a proper scoring rule, her expected 
payoff is maximized when q, = p,. Now, with q, = p,, we note that her expected payoff is: 

max{l,|}p^S(pO-Sx(qVi)] 

As S is a proper scoring rule, p,. [S(p;) — (q~,-i)] * s non-negative. Hence, her expected payoff is non- 
decreasing in b'j, and as she cannot exaggerate her budget, it is optimal for her to also report her true budget 

y t =b t . 

((U): By definition of B, agent i's payoff cannot be less than — b\. 

©I From part (Q]), the optimal strategy for i is to truthfully report her true belief p, and her true budget 
bi. As the update to q is done by mixing with i's report, the budget-constrained truthfulness property holds. 

y 

([4]): Here, we use the fact that each component of the scoring rule S is concave. Let X = max 1, By 
concavity we have, for outcome X : 

S x (q,.) = Sx((l-?i)%_i+H-) (2) 

> (1 + Wz(q«) (3) 

=>Sx(Mi)-Sx^i-i) > (4) 

Similar inequalities hold for other outcomes Y,Z, ... Note that the right hand side of Equation|4]is precisely 
the payment made to agent i by the SSM mechanism. The right hand side of the equation can be inter- 
preted as the payment that the MSR mechanism would have made to agent i if the sequence of reports was 
qi,q~2, . . . ,q~M- Thus, the worst-case loss of the SSM mechanism, over all budgets, report sequences, and 
outcomes, is no more than the worst-case loss of the corresponding MSR mechanism. 

([5]): This property also follows from equation|4] Suppose the initial market prediction is q,_i- Consider 
a set of consecutive reported distributions qu,q;2, . • . ,qi/, with corresponding budgets ba,ba,- ■ ■ ,bjj. With 
this sequence, the final reference value under SSM would be some distribution q, 7 . By equation |4j the total 
payoff for these j reports would be less than the MSR payoff of a single move to q f •. 

We will argue that an agent i with the combined budget of all j agents could make at least as high a 
profit with a single report q,-. Let b = Hb>B, then agent i could always move to q ; - -, thereby earning 

(under SSM) an amount equal to the MSR payoff of moving to q,-,-. 

If b < B, then consider an agent who believes a distribution p,. It suffices to consider the case j = 2; 
higher values of j will follow by induction, because we can first replace the last two trades with a single 

'This is a relaxation of the path-invariance property characteristic of market scoring rules, in which the sum of payoffs of 
consecutive trades is equal to the payoff of a single trade with the same final report. 
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trade without hurting profits, and then repeat this procedure. Let uq,u\,u\ denote the expected scores of 
q,-_i,qii,qi2 under this belief, and let vi denote the expected score of q ;1 under this belief. For conciseness, 
let us define X t d = ^ and A, f d = 1 — \ t . By equationlH we see that vi > X\Uq + X\U\. The total payoff of the 
2 trades is seen to be: 

Xi[u\ -M ] + X 2 [u 2 - vi] 

< — Uq] + X 2 [ll 2 — XlU\ — X\Uo] 

= \i[u\ -Uq]+X 2 [u 2 -U ] -X 2 X[[ui -M ] 

= X[X 2 [ui -U ]+X 2 [U2-U ] 

< (k\ +X 2 )max{u\ — uq,u 2 — uq} 

The right hand side of the last equation is the payoff of a single trade that reported the better of q,i , q,2 (under 
the believed distribution) with budget b = bn +b{ 2 . Thus, the single trade never has worse payoff than the 
two consecutive trades. Inductively replacing two consecutive trades with a single trade, we can show that 
this result holds for j > 2 as well. 

■ 

We note that both the strategyproofness and resistance to sybil attacks is of a myopic nature, in that it may 
not hold if agents can make trades before and after honest traders' trades. However, this restriction seems 
unavoidable, as even in the setting without budget constraints, such non-myopic attacks can be profitable 
under some models of trader beliefs lICDS+lOl . 

When we receive a forecast of q, from agent i, and know that it is rational for agent i to report this 
honestly, it may be tempting to update the market forecast to q, instead of q,. However, if we used q, as 
the reference forecast for the next trade, the market would not satisfy the myopic sybilproofness property 
described above: A trader may profit from splitting her trade into two successive trades. 

Note that, if we had updated the market forecast and reference probability 

For each agent i, if the budget b\ > is known, f s true belief can be determined from the updated market 
forecast q. (Depending on the form of Vs signal and the initial q ; _i> this may even be possible if the budgets 
are not known). Thus, in a model where the budgets are common knowledge, each trader can condition on 
all past traders' beliefs, as in the budget-unconstrained market. In other words, the SSM market generically 
enables perfect information aggregation. 

The SSM mechanism gets around the impossibility result of section |3]because it enforces a stronger re- 
striction on how an agent may use her budget. In particular, it enforces a form of the Kelly gambling [Kel56 ] 
strategy, in which an agent invests a fraction of her budget proportional to the magnitude of her uncon- 
strained move. The Kelly gambling strategy is asymptotically optimal for long-term budget growth, but it is 
suboptimal for expected short-term budget growth. In our model, this manifests itself in the following way: 
An agent is never able to stake her entire budget in her trade, and consequently, the updated prediction q~j is 
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less informative than the updated prediction of the MSR with the same budgets. However, this may be more 
than offset by the improved ability for future traders to aggregate information, and because the enforced 
proportional-betting policy leads to better long-term budget growth where agents may be acting myopically. 

6 Discussion 

Extension to Continuous Outcomes For geographical events, such as forecasting the location of a tor- 
nado strike, it may be natural to model the outcome space as continuous, and set the goal of forecasting 
the expected value. Lambert et al. BLPS081 have shown that (without budget constraints) the mean can be 
elicited by a proper scoring rule. However, Theorem [TO] extends a fortiori to a model in which we want 
to forecast the mean of a 2-dimensional outcome, with independence between the dimensions. A special 
case of the continuous model is the case in which all beliefs are over the outcomes (0,0), (0, 1), (1,0), (1, 1); 
theorem [TO] shows that no smooth scoring rule can be budget-constrained truthful in this case, and hence in 
general. 

Separable scoring rules For forecasting the mean of a multi-dimensional outcome, one natural approach 
is to elicit separate mean forecasts along each dimension, and to pay off each dimension's forecast through 
a separate scoring rule. In other words, we might have a mean latitude market and a mean longitude mar- 
ket. With such a setup, it does not matter whether an agent believes the two dimensions are independent 
or dependent; her expected payoffs would depend only on her marginal beliefs over the two dimensions. 
Theorem [TO] extends em a fortiori to this case as well: For a separable scoring rule, an agent's expected 
payoff is the same as another agent with the same marginal beliefs who believes the two dimensions are 
independent, and hence, her behavior will be the same. 

Transformed independence Because we make no additional assumptions about the structure of the scor- 
ing rule in section [4] the results extend to many families of restricted distributions. In particular, any family 
of restricted beliefs that can be smoothly reparameterized to be independent along two parameters will yield 
the same result. 

Mechanisms for risk-averse agents We have modeled traders who are risk-neutral up to a hard budget 
constraint; this can be viewed as a specific family of risk-averse utility functions. Dimitrov et al. [DSE09] 
showed that there are no attractive sequential market mechanisms to aggregate information for all unknown 
risk types: any such mechanism must exhibit exponentially reducing payoffs in order to guarantee myopic 
sybilproofness with a bounded subsidy. The scaled scoring mechanism does not have this negative property. 
Thus, we have shows one natural class of restricted risk preferences for which it is possible to sequentially 
aggregate information. We note, however, that this relies on the assumption that you cannot exaggerate your 
budget; hence, we have access to a mechanism that reveals some information about a trader's risk type. 
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Limitations and directions for future work Our impossibility results required some technical assump- 
tions on the smoothness of the scoring rules. Intuitively, there does not seem to be any essential advantage 
to using non-smooth scoring rules, and one direction for future work is to relax these assumptions. It would 
also be helpful to extend theorem |7] to higher-dimensional neighborhoods. 

The Scaled Scoring Mechanism relies on the fact that a trader cannot deposit an amount greater than 
her true budget. However, depending on her reported belief, there may be no outcome in which she loses 
her entire deposit. Thus, if there is an external credit market from which she can borrow with her trade as 
collateral, this assumption may be violated. One important question for future work is to find alternative 
market-like mechanisms for sequential information aggregation in the presence of budget limits. 
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